Medical students' perception of changes in assessments implemented during the COVID-19 pandemic.

BACKGROUND: COVID-19 posed many challenges to medical education in the United Kingdom (UK). This includes implementing assessments during 4 months of national lockdowns within a 2-year period, where in-person education was prohibited. This study aimed to identify medical school assessment formats emerging during COVID-19 restrictions, investigate medical students' perspectives on these and identify influencing factors. METHODS: The study consisted of two phases: a questionnaire asking medical students about assessment changes they experienced, satisfaction with these changes and preference regarding different assessments that emerged. The second phase involved semi-structured interviews with medical students across the UK to provide a deeper contextualized understanding of the complex factors influencing their perspectives. RESULTS: In the questionnaire responses, open-book assessments had the highest satisfaction, and were the preferred option indicated. Furthermore, in the case of assessment cancellation, an increase in weighting of future assessments was preferred over increase in weighting of past assessments. Students were also satisfied with formative or pass-fail assessments. Interview analyses indicate that although cancellation or replacement of summative assessments with formative assessments reduced heightened anxiety from additional COVID-19 stressors, students worried about possible future knowledge gaps resulting from reduced motivation for assessment-related study. Students' satisfaction level was also affected by timeliness of communication from universities regarding changes, and student involvement in the decision-making processes. Perceived fairness and standardisation of test-taking conditions were ranked as the most important factors influencing student satisfaction, followed closely by familiarity with the format. In contrast, technical issues, lack of transparency about changes, perceived unfairness around invigilation, and uncertainty around changes in assessment format and weighting contributed to dissatisfaction. CONCLUSIONS: Online open-book assessments were seen as the most ideal amongst all participants, and students who experienced these were the most satisfied with their assessment change. They were perceived as most fair and authentic compared to real-life medical training. We seek to inform educators about student perceptions of successful assessment strategies under COVID-19 restrictions and provide evidence to allow debate on ongoing assessment reform and innovation. While this work looks specifically at assessment changes during COVID-19, understanding factors affecting student perception of assessment is applicable to examinations beyond COVID-19.

Assessment during Covid-19: quality assurance of an online open book formative examination for undergraduate medical students.

BACKGROUND: The spread of COVID-19 pandemic in early 2020 compelled all the educational activities, including medical education to be shifted from face-to-face interaction to a virtual platform. This shift provided opportunities for exploring online assessment modalities. One such assessment method is an online open book exam which is a unique concept in medical education of Pakistan. Limited information is available in literature regarding open book exam for the basic science subjects. Hence, the objective of the study was to determine the quality of the open book exam administered as a pilot project to the first-year medical students. METHODS: It was a cross-sectional analytical study that included 99 students of first year MBBS. The students were administered an online unrestricted type of open book exam as a formative assessment. The exam consisted of 30 open-ended, short answer type questions. The scores of the exam were analyzed for psychometric quality. RESULTS: The mean score was 47.24 ± 15.30 SD %. The reliability of the exam was 0.79. The majority (66.6%) of items were found to be moderately difficult with their difficulty index ranging from 31 to 80%. The majority (86.6%) items were in the range of moderate to high discrimination. There were no questions with negative discrimination. CONCLUSIONS: The exam was found to be reliable and can be implemented with training of faculty and students. Online open book exam provides a good format for remote and formative assessment of students with minimum proctoring during times of constraints such as COVID-19 pandemic.

Covid-19 Catalysing Assessment Transformation: A Case of the Online Open Book Examination

Under COVID-19 lockdown conditions, the imposition of social distancing and restricted mobility, disrupted the traditional way of assessment in higher education. The closed book examination, conducted under proctored conditions, had to be substituted for the online open book examination (OOBE), posing challenges to both conventional and Open Distance Learning (ODL) institutions. The OOBE became a new experience to lecturers and students. Considering COVID-19 as a potential catalyst for educational transformation, the experiences gained in this format of assessment presents a valuable frame of reference for future learning. The aim is to extract lessons from this innovative learning experience to inform future assessment practices. The study is set in the context of a B.Ed. (Hons) compulsory module, offered at an Open Distance Learning (ODL) institution in South Africa. It is guided by the research question: "what were students' experiences of their first online, open-book final examination and what are the implications for policy, practice and research?" This is a qualitative study, using as data, student emails on their experiences of the OOBE. The results show that the OOBE is an innovative assessment practice in higher education, in need of deeper understanding and (re)training. We conclude that the OOBE offers transformational opportunities in higher education assessment practices, to replace the traditional closed-book examination. We make recommendations to assist lecturers and students in approaching the OOBE in future.

MATLAB and Python Open Book Assessments: Lessons from Two UK Institutions

Although programming has increasingly become an integral part of Mathematics education in universities, the closed book exam, which excludes programming, remains the preferred main assessment component. However, the switch towards open book online assessments necessitated by the Covid pandemic has seen programming content being included in end of year assessments. In this paper we look at how programming has been integrated into the end of year open book online assessments in Mathematics at two UK universities. Preliminary conclusions suggest that open book online assessments incorporating programming content can be a viable replacement for the traditional closed book exam. © 2022 IEEE.

Rethinking Assessment: The Future of Examinations in Higher Education

The global higher education landscape is significantly impacted as a result of the COVID-19 pandemic and the majority of the universities now follow an online or hybrid mode of delivery. This presents substantial challenges for universities, particularly to conduct examinations, as traditionally most exams were conducted physically on campus. During the first wave of the pandemic, many universities had no option and were forced to move online in a very short period of time, causing universities also to conduct exams online without transforming pedagogy and the structure/s of closed-book exams. Inevitably, in non-proctored and unregulated examinations, this allowed room for students to collaborate and share material during online exams without being noticed by an invigilator as in the case of physical exams. Online exams, also leave room for students to find information online which made preventing plagiarism a significant challenge. This paper investigates the practices used in both closed-book and open-book exams and identifies the challenges associated with the transition to online exams. It also identifies potential ways forward for future online exams, while minimizing opportunities for students to collaborate, plagiarise and use online material. The findings of this study reveal that online examinations affect teachers and students differently: while teachers have mixed feelings about online exams, students are anxious about their grades and the technical hassle they experience in online exams. While viva has emerged as a popular form of alternative assessment, students still feel the need of returning to physical exams. None of the teachers who participated in this study discussed a psychosocial approach to education and exams in this pandemic. We conclude this study on the note that there is a need for the collaboration of social scientists, psychologists, psychosocial specialists, educationists, and humanities scholars/humanists for better educational policy and pedagogical practices during the pandemic.

Analytics of Open-Book Exams with Interaction Traces in a Humanities Course

Open book exams (OBE) have been a mandated part of each course structure at some universities. Also during the COVID19 emergency remote teaching situation, OBE would be an option for many instructors over a proctored examination. In this study we investigate a Critical Analysis of Literature and Cinema course which had offered open book exam components for more than 11 years in a face-to-face classroom mode. However, this time the OBE was conducted online using BookRoll, a learning analytics enhanced eBook platform. 89 Students accessed Hayavadana, an Indian play uploaded on BookRoll during the exam. They attempted a critical reading task to identify performative elements and cultural references in the text by highlighting them with yellow and red markers respectively and writing a reflective memo about the identified items in BookRoll. We analyzed learner’s interaction logs gathered in the learning record store linked to BookRoll during the OBE and investigated the relations between their critical reading behaviors to the OBE achievement. Further, selecting two distinct achievement groups we conducted process mining to identify distinct reading behaviors of the high and low performers and give examples of their generated reflective memos. This study aims to initiate further discussion related to the application of learning analytics in humanities courses and probed into the behaviors of the students during the OBE. © 2021 29th International Conference on Computers in Education Conference, ICCE 2021 - Proceedings. All rights reserved

Combating Academic Dishonesty in Higher Education Exams during the COVID-19 Pandemic: The Role of Open Book Exams

Background: One of the clear effects of the spread of the Covid-19 virus on educational work is the shift to remote teaching and assessment. Since the switch to remote exams, it became clear that the biggest challenge facing electronic exams is the spread of academic dishonesty to the extent that it has become a concern for teachers, educators, officials and parents. The preoccupation of these individuals became the need to search for what would put an end to this problem.Objective: This research aims to shed light on the open book exams that can contribute to reducing this problem. It also aims to show the role of the open book exam in reducing academic dishonesty in electronic exams.Methodology: In the methodology, the rapid review method was done for collecting data. Papers were then categorized and analyzed to realize the goals of this research.Results: It has been found that electronic exams suffer from an outbreak of academic dishonesty. And the use of the open book exam, especially the one that is built in a sound way, reduces the phenomenon of academic dishonesty.Conclusion: The use of open book exams contributes positively to the reduction of academic dishonesty provided that the exam is designed and built appropriately and both the lecturer and the student are well trained to use this type of exam. © 2021 IEEE.

The relative impact of in-class closed-book versus take-home open-book examination type on academic performance, student knowledge retention and wellbeing

This study investigates the relationship between take-home (open-book) examinations (THE) and in-class (closed-book) examinations (ICE) on academic performance and student wellbeing. Two social science courses (one bachelor and one master) were included in the study. In the first cohort (2019), students from both courses performed an ICE, whereas students in the second cohort (2020) performed a THE. Four to six months following course completion, students were approached to fill out a survey pertaining to their academic performance and wellbeing during the course, and to complete a test measuring knowledge retention on the course materials. No significant differences were found between the ICE and THE cohorts in academic performance and knowledge retention for either the bachelor or the master students. Bachelor students who completed a THE in 2020 reported significantly lower wellbeing in comparison to their peers completing the ICE a year earlier. The implications of the results in the context of the Covid-19 pandemic are discussed.

Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination.

Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom's taxonomy ('Remember', versus 'Understand/Apply'). We predicted higher performance on the OBE, driven by higher performance on 'Remember' questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms 'Open-Book, 'Bloom Category' and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14-2.39), and higher on 'Remember' than 'Understand/Apply' questions (OR 1.13, 95% CI: 1.09-1.19). The difference in performance between 'Remember' and 'Understand/Apply' questions was greater in the OBE than the CBE ('Open-Book' * 'Bloom Category' interaction: OR 1.2, 95% CI: 1.19-1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge.

Challenges of unrestricted assignment-based examinations (ABE) and restricted open-book examinations (OBE) during COVID-19 pandemic in India: An experimental comparison.

COVID-19 pandemic has affected every sphere of life specially the education sector observing a paradigm shift in the nature of pedagogy from offline face-to-face to online-virtual mode of learning. The biggest challenge in online-learning was the conduction of online examination for student's assessment specially in Indian context where digital divide is rampant. Thus, present study examines and compares the challenges faced by the students in two most widely accepted modes of examination by Indian universities and institutes of higher learning, that is, take home/unrestricted/assignment-based examination (ABE) and highly time restricted/open-book examination (OBE). Primary data was collected through questionnaires prepared by using Google forms to measure adaptability, satisfaction, and challenges using 5-point Likert's scale. Cronbach's α test was performed on question items to check the reliability and internal consistency of the items. χ 2 test has been applied in order to check whether there is a statistically significant relationship between the gender and place of residence in the acceptability of ABE and OBE. The findings suggest that both modes of examination have their own challenges largely governed by the digital and economic divide. The acceptance level of ABE and OBE is not associated with gender. However, we found the level of acceptance association of ABE with the place of residence of the students but not with OBE.

Switching from face-to-face to an online teaching strategy: how anatomy and physiology teaching transformed post-COVID-19 for a university preprofessional program.

The College of Science and Health Professions offers a university preprofessional program. Like most medical schools in Saudi Arabia, the teaching delivery strategy in the university preprofessional program is on campus and face-to-face. During the month of March 2020, teaching activities of the spring semester were proceeding as normal; however, the sudden emergence of COVID-19 disturbed routine activities and compelled authorities to switch all teaching activities from face-to-face to online. Training sessions and workshops for all stakeholders on online delivery methods were arranged. Blackboard and other online facilities were utilized. All teaching materials, including newly made video clips for anatomy and physiology practicals, were uploaded on Blackboard and discussed online with students. Students' anxiety related to the exam was reassured by giving them the option of open book quizzes during summative continuous assessment. All scheduled teaching sessions, lectures, and practicals were conducted proficiently. Revision sessions and assessment quizzes were conducted with students' satisfaction. At the end of the semester, a final exam was conducted online as an open book exam. Students with technical issues while attempting the exam were given an opportunity to make up for it. After a successful final exam, the cumulative block grades showed students secured higher grades in the open book exam. Following that, the King Saud bin Abdulaziz University for Health Sciences has managed to conduct on-campus close book exams that abide by self-distancing and standard operating procedure policies.

Does a sudden shift of testing format from closed-book to open-book change the characteristics of test scores on summative final exams?

INTRODUCTION: In response to the COVID-19 pandemic, most universities in North America transitioned to online instruction and assessment in March 2020. Undergraduate pharmacy students in years one to three of two four-year entry-to-practice programs at a university in Canada were administered open-book examinations to complete their didactic winter-term courses in pharmaceutical sciences; behavioural, social, and administrative sciences; and pharmacotherapeutics. The impacts of the switch to open-book examinations on final exam characteristics are examined. METHODS: The ratios and correlations of final exam and midterm grades in 2020, where final exams were open-book, and in 2019, where finals were closed-book, were calculated and compared. RESULTS: In 2020, the ratio of final exam to midterm exam scores for five out of seven courses were significantly larger than they were in 2019. Alternatively, for all but one course, the correlations between midterm and final examination grades showed no significant difference from 2019 to 2020. CONCLUSIONS: Compared to 2019 when finals were administered in a closed-book format, a sudden shift to an open-book format for final exams in 2020 appears to be associated with the final exams becoming easier relative to midterms. However, when considering how final and midterm exam grades correlate year over year, in all but one class, there was no significant difference. These findings suggest that changing exams to be open-book may change how they can be used to inform criterion-referenced or absolute grading decisions but not norm-referenced or rank-based decisions.

Effect of remote online exam delivery on student experience and performance in applied knowledge tests.

BACKGROUND: The use of remote online delivery of summative assessments has been underexplored in medical education. Due to the COVID-19 pandemic, all end of year applied knowledge multiple choice question (MCQ) tests at one UK medical school were switched from on campus to remote assessments. METHODS: We conducted an online survey of student experience with remote exam delivery and compared test performance in remote versus invigilated campus-based forms of similar assessments for Year 4 and 5 students across two academic years. RESULTS: Very few students experienced technical or practical problems in completing their exam remotely. Test anxiety was reduced for some students but increased for others. The majority of students preferred the traditional setting of invigilated exams in a computer lab, feeling this ensured an even playing field for all candidates. Mean score was higher for Year 4 students in the remotely-delivered versus campus-based form of the same exam (76.53% [SD 6.57] vs. 72.81% [6.64]; t438.38 = 5.94, p = 0.001; d = 0.56), whereas candidate performance was equivalent across both forms for Year 5 students. CONCLUSIONS: Remote online MCQ exam delivery is an effective and generally acceptable approach to summative assessment, and could be used again in future without detriment to students if onsite delivery is not possible.

Medical Teaching and Assessment in the Era of COVID-19.

The coronavirus pandemic has had a significant influence on medical education, most notably in terms of content delivery and provision of assessments. These unique times have facilitated the introduction of many new educational methods in medical schools globally each of which with potential merits and drawbacks. Importantly, the use of remote platforms to carry out online teaching has been especially vital in ensuring the continued training of doctors but other techniques such as telemedicine, online clinical case repositories and even virtual reality headsets are also being used to overcome these difficult circumstances. Some institutions also opted for open book written examinations raising issues surrounding this format's legitimacy and potential benefits. Practical examinations are even harder to facilitate and although most were cancelled, this crisis may result in innovation which changes their future format. For example some may include audible clinical signs played online through a computer speaker to replace clinical examination. Overall, the COVID-19 pandemic has caused an overhaul of medical school orthodoxy that whilst disruptive, may serve to expose institutions to novel means of teaching and assessment which may ultimately improve medical education in future.

Evaluation of academic integrity of online open book assessments implemented in an undergraduate medical radiation science course during COVID-19 pandemic.

INTRODUCTION: Online open book assessment has been a common alternative to a traditional invigilated test or examination during the COVID-19 pandemic. However, its unsupervised nature increases ease of cheating, which is an academic integrity concern. This study's purpose was to evaluate the integrity of two online open book assessments with different formats (1. Tightly time restricted - 50 min for mid-semester and 2. Take home - any 4 h within a 24-h window for end of semester) implemented in a radiologic pathology unit of a Bachelor of Science (Medical Radiation Science) course during the pandemic. METHODS: This was a retrospective study involving a review and analysis of existing information related to the integrity of the two radiologic pathology assessments. Three integrity evaluation approaches were employed. The first approach was to review all the Turnitin plagiarism detection software reports with use of 'seven-words-in-a-row' criterion to identify any potential collusion. The second approach was to search for highly irrelevant assessment answers during marking for detection of other cheating types. Examples of highly irrelevant answers included those not addressing question requirements and stating patients' clinical information not from given patient histories. The third approach was an assessment score statistical analysis through descriptive and inferential statistics to identify any abnormal patterns that might suggest cheating occurred. An abnormal pattern example was high assessment scores. The descriptive statistics used were minimum, maximum, range, first quartile, median, third quartile, interquartile range, mean, standard deviation, fail and full mark rates. T-test was employed to compare mean scores between the two assessments in this year (2020), between the two assessments in the last year (2019), between the two mid-semester assessments in 2019 and 2020, and between this and last years' end of semester assessments. A p-value of less than 0.05 was considered statistically significant. RESULTS: No cheating evidence was found in all Turnitin reports and assessment answers. The mean scores of the end of semester assessments in 2019 (88.2%) and 2020 (90.9%) were similar (p = 0.098). However, the mean score of the online open book mid-semester assessment in 2020 (62.8%) was statistically significantly lower than that of the traditional invigilated mid-semester assessment in 2019 (71.8%) with p < 0.0001. CONCLUSION: This study shows the use of the online open book assessments with tight time restrictions and the take home formats in the radiologic pathology unit did not have any academic integrity issues. Apparently, the strict assessment time limit played an important role in maintaining their integrity.

A novel approach to medical school examinations during the COVID-19 pandemic.

Online teaching for medical students is not an unusual tool used in medical education. Alongside clinical placements, medical students are familiar with online teaching platforms from various members of the faculty. However, the new and necessary method of examining medical students from their own home during the Covid-19 Pandemic is a novel approach. It is vital that medical students continue to be examined, as this establishes the attainment of the curriculum learning outcomes.